The Newtonian fluid test fails for three procs, but not for one and four. The tolerances all look reasonable, so not sure why this test fails now only with three procs. We could try recomputing the test with a direct solver to rule out any linear solver problems?

@mrp089 I am not sure I know how to use a direct solver on svFSIplus. For macOS we do not have PETSc or Trilinos libraries, so unless we have a direct solver implemented in svFSILS, I am not sure how to use it.

If our PETSc interface can read configuration files, it's pretty easy, e.g.:

-ksp_type preonly
-pc_type lu
-pc_factor_mat_solver_type mumps

If not, we first would need to add a line or two to read input files.

@mrp089 this is one point we do not read options from file. The main point is for macOS testing we do not use petsc or trilinos. We do not build the solver with them, so we don't have them available.

Oops, I missed that part in your reply, sorry. We could still recompute the reference solution on Ubuntu with a direct solver. Or we could tune up the tolerances of the linear solver more (if the error goes lower) in the reference solution and in the test and see if that fixes it. As a last resort, coarsen the test tolerance

@dcodoni Some comments from a casual observer

• Why is the bipartition solver not being used for a fluid simulation ?
• The tolerances are much too small, 1e-5 tolerances are typical
• The mesh is pretty coarse

It turns out that this pipeline has different convergence behavior for 1 (good), 3 (bad), 4 (good) procs for test_newtonian:

 test_newtonian[1]
 NS 1-1  1.390e-01  [0 1.000e+00 1.000e+00 9.108e-13]  [114 -277 19]
 NS 1-2  2.670e-01  [-82 7.699e-05 7.699e-05 2.604e-09]  [89 -198 13]
 NS 1-3  3.870e-01  [-176 1.458e-09 1.458e-09 1.312e-04]  [69 -89 9]
 NS 1-4s 5.050e-01  [-254 1.945e-13 1.945e-13 1.000e+00]  !0 0 0!  WARNING: The linear system solution has not converged

test_newtonian[3]
 NS 1-1  7.600e-02  [0 1.000e+00 1.000e+00 8.948e-13]  [114 -277 17]
 NS 1-2  1.760e-01  [-82 7.699e-05 7.699e-05 2.886e-09]  [201 -197 37]
 NS 1-3  2.610e-01  [-68 3.774e-04 3.774e-04 6.289e-10]  [155 -212 27]
 NS 1-4s 3.370e-01  [-119 1.037e-06 1.037e-06 1.886e-07]  [80 -155 11]

test_newtonian[4]
 NS 1-1  5.900e-02  [0 1.000e+00 1.000e+00 9.066e-13]  [114 -277 19]
 NS 1-2  1.140e-01  [-82 7.698e-05 7.698e-05 2.893e-09]  [87 -197 13]
 NS 1-3  1.660e-01  [-176 1.444e-09 1.444e-09 1.292e-04]  [69 -90 10]
 NS 1-4s 2.190e-01  [-254 1.912e-13 1.912e-13 1.000e+00]  !0 0 0!  WARNING: The linear system solution has not converged

@dcodoni, quick fix: set Max_iterations to 10 and see if it passes

@mrp089 I did and it passes! It is converging in 7 iterations, it seems stiffer for the first 3 iterations repsects to the 1proc and 4procs case.

I'm good with the quickfix in the interest of moving this and other PRs along. I documented the problem in #284, and we can keep digging into it over there.

The primary goal of CI is to ensure that the code works as expected after code modifications. Three things should ideally be checked

• results match gold standard results
• execution times are similar
• solver behavior is the same: convergence criteria matches

We really do need to understand what the linear algebra settings (e.g. number of maximum iterations, tolerances, etc.) are testing and to make sure that all CI tests are consistent.

@mrp089 Thank you Martin for opening the issue.
@ktbolt I think you are right, but I do not have any idea on how to make tests more consistent. But I think that at least each physics should be solved with the most appropriate linear solver (CG for structure, GMRES for FSI, BIPN for fluid).